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The appearance of agile software development techniques (ASDT) since 
2001 has encouraged many organizations to move to an agile approach. 
ASDT presents an opportunity for researchers and professionals, but it has 
many challenges as well. One of the most critical challenges is agile effort 
prediction. Hence, many studies have investigated agile software 
development cost estimation (ASDCE). The objective of this study is 
twofold: First, to propose an improved model based on support vector 
regression with radial bias function kernel (SVR-RBF) enhanced by the 
optimized artificial immune network (Optainet). Second, to perform a 
detailed comparative analysis of the proposed method compared to other 
existing optimization techniques in the literature and applied for ASDCE. 
The experimental evaluation was carried out by assessing the performance of 
the proposed method using some trusted measures like standardized 
accuracy (SA), mean absolute error (MAB), prediction at level p (Pred(p)), 
mean balanced relative error (MBRE), mean inverted balanced relative error 
(MIBRE), and logarithmic standard deviation (LSD). Throughout a dataset 
with 21 agile projects using the leave-one-out cross-validation (LOOCV) 
technique. The results obtained prove that the proposed model enhances the 
accuracy of the SVR-RBF model, and it outperforms the majority of existing 
models in the literature. 
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1. INTRODUCTION 


In the past twenty years and since the announcement of twelve principles of the agile manifesto in 
2001, researchers have shown an increased interest in agile software development. Agile is a generic term 
that encompasses different methods, approaches, practices, and techniques that meet manifesto principles and 
values [1]. The agile software development techniques (ASDT) have many attractive advantages like the 
incremental and iterative aspect of development, quick releases, and good adaptation to the volatile nature of 
customer requirements. Among the well-known ASDT, we found feature-driven development [2], eXtreme 
programming [3], and the scrum framework [4]. Nevertheless, for all these ASDT, one of the most severe 
challenges is to produce an initial reliable and accurate estimate of the project to be developed. 

The primary element to estimate the needed development effort is the project size. Principally, story 
point is a highly employed measurement to deduce the size and the complexity of an agile project. Besides, 
the velocity factor refers to the number of story points conveyed by the team in a sprint. It significantly 
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impacts the development effort of agile projects. Thus, we used these two measures to predict the effort 

needed using ASDT. 

Generally, cost estimates have an important impact on the process of software development [5]. In 
fact, accurate estimation leads to the success of projects, while inaccurate predictions lead to the failure of 
projects [6]. For example, underestimation results in less budget and fewer resources leading to a project with 
low quality that fails to be accomplished within estimated deadlines. While overestimation leads to expecting 
more resources than what is needed. Therefore, seeking an efficient technique that produces accurate 
estimates is useful and of high importance. Especially with the study reported by the standish-group- 
international (SGI) [7], where more than sixty percent of projects partially or completely fail (19% failed, 
44% remains challenged, and only 37% successful). 

According to Britto et al. [8], various algorithmic and non-algorithmic techniques have been used 
for agile software cost estimation (ASDCE) purposes. Unfortunately, the accuracy of effort estimations is 
still a significant challenge. Moreover, it did not achieve acceptable estimation accuracy. This result has been 
also confirmed by the systematic literature review conducted by Fernandez-Diego et al. [9]. In the same 
study, the authors reported some accuracy improvement; however, many papers present unacceptable 
estimations. This is why there is still a need to enhance accuracy by proposing new improved ASDCE 
models or optimizing the existing ones. 

Furthermore, prediction techniques couldn’t be assessed without applying suitable accuracy 
measures. According to [9], the most commonly used measures are based on the median or the mean of 
magnitude of relative error (MMRE, MdMRE), with 35 studies from 73 and Pred (25%). However, 
considering the inconvenience of MMRE and its sensitivity to outliers, further measures like the percentage 
of accuracy, mean absolute error (MAE), and R2 have been used in conjunction to raise the appropriateness 
of the accuracy. That’s why our second goal from this study is to assess the performance of our proposed 
model using some trusted criteria as recommended in many papers in the literature like in [10]-[12]. 

Up to now, some studies have investigated the use of the support vector regression (SVR) model for 
agile see [13]-[17] thanks to its power in dealing with the agile software effort estimation concerns. 
However, an optimal SVR model is still needed. In this sense, Zakrani et al. [17] have proposed an optimized 
SVR model using a grid search (GS) method to tune the hyperparameters of the SVR model where every 
hyperparameter combination is evaluated. This exhaustive search is perceived as inefficient, the time will 
exponentially grow according to the dimension of evaluations. Also, the population size is user-defined and 
not adjustable which affects the efficiency of the algorithm and might not find the true optimum. 

This present study proposes an improved SVR model with RBF (SVR-RBF) kernel for ASDCE 
based on the Optainet algorithm. The Optainet [18] is used to tune the SVR-RBF hyperparameters to find the 
best configuration model that gives accurate estimates. The Optainet explores the effectiveness of the aiNet 
theory, which simulates the human immune system activity (it has a robust memory and a remarkable ability 
to differentiate the self cells from the foreign ones). The Optainet algorithm has many advantages regarding 
the GS method: 1) Inclusion of stopping criteria which controls the time of execution, ii) Ability to check and 
maintain various optimal solutions, iii) Capacity to exploit and explore the whole space of search, and 
iv) The capability of dynamic adjustment for the population size. 

Our major objectives through this study are: 

— We suggested an improved SVR-RBF model using the optimized artificial immune network to seek 
optimal parameters of the SVR-RBF model. This proposed hybrid model enhances the performance of 
the estimations. 

— We applied a very known and trusted empirical evaluation protocol using the following criteria: Pred, 
LSD, MIBRE, MBRE, and MAE. 

— We performed a robust comparative empirical study to assess the suggested model's accuracy compared 
to the accuracy of other techniques. 

To synthesize the goal from this study, we aim during our analysis to answer the following three questions 

(Qus): 

—  Qul: Does the Optainet enhance the performance of the predictions produced using SVR-RBF? 

—  Qu?2: How are the estimations of the SVR-RBF using Optainet in comparison with other optimization 
methods? 

—  Qu3: Is the performance of our model more accurate than the other models? 

The remainder of this study is organized in: section 2 highlights the previous ASDCE-related works. 
Section 3 describes the suggested model and the used optimization technique. Section 4 presents the overall 
experimental evaluation while the obtained results are outlined in section 5, and section 6 reports the 
conclusion. 


Int J Artif Intell, Vol. 11, No. 1, March 2022: 265-275 


Int J Artif Intell ISSN: 2252-8938 ) 267 


2. RELATED WORKS 

Multiple papers have addressed ASDCE problems. For example, in [9], 73 relevant studies have 
been identified in a systematic literature review (SLR) on ASDCE papers published between 2014 and 2020. 
This SLR presents an update of Britto et al. [8] SLR study on ASDCE methods published between 2001 and 
November 2013 (where 25 papers were selected). According to [8], various techniques have been used for 
ASDCE purposes, but the most used techniques are those qualified as expert-based prediction techniques. 
Moreover, most methods did not achieve an acceptable estimation accuracy regarding the observed 
difference between the real effort value and the predicted one. Notably, the accuracy achieved by the 
majority of studies did not meet the suggested 25% threshold indicated in [19] for assessing the accuracy of 
cost prediction. For example, there are commonly accepted thresholds for MMRE and Pred measures, i.e an 
acceptable MMRE value should be under or equal to 25%, and an acceptable Pred (0.25) should be above or 
equal to 75%. 

In [9], six ASDT were identified: extreme programming (XP), Scrum, feature-driven development, 
distributed agile software development, Kanban, and agile unified process. Overall, expert-based techniques 
played a significant role in cost prediction such as the planning poker method, which is intently related to 
story points sizing metric. Besides, the estimation accuracy challenge, some studies report a suitable value of 
accuracy, while many papers stated insufficient precision. Also, about 29% of studies used the accuracy 
measures, while nearly 18% of studies used effect size metric to compare methods. Although ASDT remains 
dependent on expert judgment (notably, 24.66% of studies used the planning poker method), it is revealed 
that there has been a significant increase in the number of papers using machine learning techniques, exactly 
20.55% of papers [9]. Next, we briefly introduce some examples. 

In [20], the authors suggested a model for ASDCE based on user stories, which is adapted to the 
agile specificity (iterative and adaptive methodology). The empirical evaluation was made based on 21 
software agile projects and using MMRE and Pred accuracy measures, and the results show acceptable effort 
estimations values based on the two metrics. The work in [21], proposed a new formula using story point and 
velocity variables to estimate the cost. The combination of two optimization techniques, namely particle 
swarm optimization (PSO) and bee colony, was employed to determine and optimize the formula’s 
parameters. The results revealed that the proposed technique performs better than the models proposed in 
[20], [22] based on MAR, R2, Pred (8%), MdMRE, and MMRE. 

In [14], the authors attempted to enhance the prediction of ASDT through the use of SVR with 
several kernels. The model is then evaluated using 21 agile projects. The obtained results demonstrate that 
SVR-RBF outperforms the other models based on MMRE and Pred (25%) measures. In the same vein, 
Zakrani et al. [17] proposed an improved version of the SVR-RBF model, which uses the grid search 
optimization method to tune the key hyperparameters. The experimental evaluation was made based on 21 
agile projects, and then the model was validated using the LOO-CV method. The results showed that the 
accuracy of SVR-RBF was improved, and it outperformed other studies in terms of MAMRE, MMRE, and 
Pred (25%). 

Nonetheless, the accuracy reported in the previous studies still needs further enhancement and a 
robust comparative study based on reliable accuracy measures and a significant statistical test. This is exactly 
what motivated our present study where we optimize SVR-RBF by Optainet and conduct a comprehensive 
comparative analysis. The following subsections provide more details about the proposed model. 


3. THE PROPOSED METHOD: SVR-RBF-OPTAINET 
3.1. SVR model: an overview 

The support vector machine (SVM) is a powerful machine learning (ML) model that has offered 
valuable results in both classification and regression problems [23]. The SVM approach has several attractive 
features like the sparse way of presenting solutions, good generalization ability, and the capacity to avoid 
local minimums, thanks to the structural risk minimization concept. In this paper, we employ s-SVR that 
introduces the g-insensitive loss function as shown in Figure 1. It means that all errors less than a defined 
threshold (inside bars in the figure) are neglected, while errors induced by the points located out of the bars 
are calculated using y and w* like in Figure 1. In the case of nonlinear regression, the following (1) is used 
where @ denotes a nonlinear function that maps the low input space to the high output space; w represents the 
weights vector, and c is the threshold. 


h(x) = wl (x) +e (1) 


We mention that w and c are selected for optimizing the next problem [24]: 
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wy, ¥7,= 0. (2) 


Figure 1. The ¢SVR regression 


The ¢ represents the function’s deviation, the regularization parameter B reflects the trade-off 
between the error’s tolerance above ¢ and the flatness of h, the y, y* as proposed in [25] presents the slack 
variables that define the tolerated deviations above the error ¢. The base mechanism of the SVR model is 
minimizing the objective function which takes into account the norm of w and the loss of w;, and Y/‘as 
expressed in (2). We note that the Lagrangian multiplier is used in the SVR method, it is related to the dot 
product of @ (x). It can be realized through the kernel function determined as: H(xi, xj)=<@(xi), @(xj)> which 
prevents the explicit computation of the @ (x). We refer the reader to [24] for more details. In the present 
study, we used the SVR-RBF towing to its ability to generate good estimates [11], [12]. The computation of 
the SVR-RBF kernel is done via the expression: H(xi, xj)}=exp(—A||xi-x\|’). Hence, parameter 4 should be 
cautiously chosen beside the B and the € parameters. 


3.2. Optainet algorithm 

The artificial immune network (aiNet) model was introduced by Castro and Zuben [26]. It is a graph 
composed of nodes (antibodies) and edges. It is essentially based on the immune system (IS) theory [27] 
where the IS generates many antibodies or attempts to provide the best-suited antibodies for attacking 
antigens. The modelization of this phenomenon corresponds to a function optimization process. Many 
algorithms inspired by the biological domain and especially by the IS were used to achieve global 
optimization like the opt-IA [28], the B-Cell algorithm [29], the opt-aiNet [18], and the Clonalg algorithm 
[30]. 

In this work, the SVR-RBF with Optainet optimization [18] is used to optimize the SVR-RBF 
parameters. The Optainet explores the effectiveness of the aiNet theory, which simulates the human immune 
system activity (it has a robust memory and a remarkable ability to differentiate the self cells from the foreign 
ones). The Optainet algorithm has the following advantages: 

— Inclusion of stopping criteria. 
— Ability to check and maintain various optimal solutions. 
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— Capacity to exploit and explore the whole space of search. 

— The capability of dynamic adjustment for the population size. 

We adopt the following terminology to explain the Optainet algorithm: 

— Cells: Each cell in the network is composed of population values. In the Euclidean space, it is denoted 
with a multivalued vector. 

—  Cell’s: Fitness stands for the objective function value using a particular network cell. 

— Cells: Affinities represent the value of distance (Euclidean) that exists between two cells. 

— Cloning: Cloning is producing replication of original cells (or parents). 

— Mutation: The generated copies (or offspring) will be mutated to be different from their parents 
(mutation). 

The overall Optainet steps are described as noted below: 

BEGIN 

a) Randomly initialize the population. 

b) If the stopping criteria are not satisfied, do: 

— Calculate the fitness of each cell and evaluate it according to the used objective function. Then proceed 
to its normalization. 

— Perform cell cloning according to the chosen Nc, which represents the number of offsprings 
(or clones). 

— Performing the mutation operation, where each clone will be inversely proportional to its parent fitness 
value as expressed below: 


cl’ =cl+oN(0,1) (3) 
o =(1/n)exp(-h*) 


where N(0,/) presents the random gaussian variable with standard deviation equal to 1 and a zero mean, 
cl is the parent cell, and the cl’ is the cell resulting from the mutation of cl, also h* €[0,1] represents the 
normalized fitness of a cell, while the parameter n controls the exponential function’s decay. 
Acceptable mutations are those that are within the domain interval. 

— Calculate the fitness of each cell present in the population (including cloned and mutated ones). 

— Select the highest fitness cells per clone and exclude the others. 

— Evaluating all cell affinities, then suppressing the cells having affinities lower than a predetermined 
suppression minimum os (threshold). 

— Back into the second step, after adding a p% of random cells. 

c) Else, selecting the highest fitness cell and Ending the algorithm. 

END 


3.3. Optainet based SVR-RBF model 
3.3.1. Cell design 

We used the e-SVR model with RBF kernel to implement the proposed model. So, the parameters 
(B, A, €), should be cautiously chosen and carefully optimized using our suggested Optainet-based model. 
For that reason, each cell in the network includes four elements: the B, A, ¢, and features bits. Figure 2 shows 
the cell design representation. The three first parts represent the SVR-RBF parameters. 


B, Ay us 


Figure 2. The cell with three parts, the B, A, € 


3.3.2. Fitness function 

Various criteria were suggested and employed in the literature to evaluate the performance of 
software cost prediction models. While Pred, MMRE, and MdMRE represent the frequently used ones in 
many studies [31]. For example, Braga et al. [32], and Shin and Goel [33]. Therefore, in this study, we 
employ a fitness function based on two criteria (4) where we excluded MMRE because it is susceptible to 
bias. An acceptable value for Pred (25%) is more than or equal to 75%, while for MMRE, MdMRE values 
must be less than or equal to 0.25. So, we look for cells with higher values of Pred and small values of 
MdMRE. 
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Fitness_function = (100 — Pred(25%)) + MdMRE (4) 
3.3.3. SVR-RBF-optainet design 


We describe in this subsection the architecture of our model for parameter optimization. Especially, 
the design of the SVR-RBF-Optainet model. Figure 3 shows the main steps of our proposed models. 


Network 
population 


Training 
set 


Training SVR-RBF 
Model 


Trained SVR-RBF 
Model 


Testing set 


Parameters 


Cloning and Mutation 


Fitness Evaluation 
& affinity 
measurements 


Adding new Stopping criteria 
cells to the are satisfied 
network 


Selecting best cell: 
with Optimized END 
(B, 4, €) 


Figure 3. Design of the SVR-RBF-Optainet model 


The main steps of our proposed model are described next: 

— Data scaling: We firstly proceed to the data-preprocessing phase, where we normalize all the available 
data in the range between 0 and | to strengthen the accuracy of the regressor. The expression used for 
normalization is noted below: 

* Y-Ymin 
- nin 5 
y Ymax—Ymin ( ) 
where Ymax 1S the maximal data value in the dataset, ymin is the minimal data value, y is the original 
value, and y* is the value after normalization. 
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— After that, the main steps of the Optainet algorithm were performed: Cloning, mutation, fitness 
evaluation, affinity measurement, and network suppression as shown in Section 3.2. The output of this 
algorithm is finding the best solution with parameter optimization. 

The Optainet algorithm seeks the optimal SVR-RBF configuration by exploring the parameters 
search space provided in Table 1. This is done by respecting the fitness function. Hence, the optimal solution 
that maximizes the fitness function is then chosen. 


Table 1. The values of SVR-RBF model and Optainet algorithm 
Methods Hyperparameters 
Optainet Number of iteration={500} 
Size of population={50} 
Number of clones={20} 
The threshold of suppression={0.8 } 
Newcomers percentage={50% } 


SVR Type={eps-regression} 
Kernel function={RBF} 
Range values of Hyparameters 
Complexity=[1000, 1500, 3000, 4000, 5000] 
Kernel parameter=[0.9, 1,2] 
Epsilon=[0.01, 0.02,0.03] 


4. EMPIRICAL DESIGN 
4.1. Description of dataset 

We employ a dataset that includes twenty-one agile projects with two features: the velocity and the 
story points. A summarized statistical description is presented in Table 2. It includes the number of features, 
the dataset size, min, max, mean, median, kurtosis, and skewness of cost. According to kurtosis and skewness 
values, we notice that the distribution of the cost is not normal besides the weak existence of outliers. 


Table 2. Descriptive statistics of the dataset 


Elements Values Cost 
Dataset size 21 Mini Maxi Mean Median Skewness Kurtosis 
Features’ number 2; 21 112 56.43 52 0.6049 -0.7679 


4.2. Performance measures and validation method 

Concerning the validation method, we employ the LOOCV. The LOOCV is chosen for multiple 
reasons: it is qualified as a deterministic method, unlike other techniques of validation [34], it produces 
estimations with large variance and less bias [35]. The LOOCV process is made by excluding one project 
(test project) from the dataset and performing the estimations with the rest (training set). 

Miyazaki et al. [36] proposed two accuracy metrics: the mean inverted balanced relative error 
(MIBRE), and the mean balanced relative error (MBRE) (7) and (8) accordingly. The authors claim that these 
proposed measures are insensitive to asymmetry and bias contrary to the MMRE criteria (11) which is 
vulnerable to bias and outliers. Additionally, Shepperd and MacDonell [10] have proposed a novel accuracy 
criterion, which is the standardized accuracy (SA) (14) which relies on the MAE (13). The SA indicates if the 
estimated method is better in comparison to randomly guess (PT,). Therefore, and as reported in [10] a value 
close to one means that the used method performs better than random guesses. Where MAEp,, represents the 
average of a large number of randomly guessing runs. In particular, we used 1000 for this work, MAEpr, 


represents the average of the absolute error of a specific estimation method i. 


AbE; = lyi — Hil (6) 
MIBRE = aera (7) 
MBRE = ieee (8) 
MRE = “i (9) 


Vi 
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on) Lyn (Lif MRE; < 25% 
predaes) N Bh { 0 otherwise (19) 
MMRE = = hg MRE (11) 
MdMRE = median(MRE) (12) 
1 
MAE = +'"9 ABE; (13) 
Cy ae eee (14) 
MAE pT) 
yn (6,4)? 
LSD = [fet 2 (15) 
n-1 
6; = InQ) — In (KH) (16) 
on MAR-MAR pT, (17) 


SPT 


In this present paper, we used an additional criterion (15): the logarithmic standard deviation (LSD) 
to assess the performance of estimation methods, where y; presents the actual cost while ¥, is the i™ project’s 
estimated cost, and z” is the estimation related to the 5; variance (16). Besides, we used the effect size metric 
(A) as expressed in (17) to test if the generated estimates are produced by chance or not. The SPT) represents 
the standard deviation of randomly guessing. The interpretation of the values of (A) as explained in [37] is: 
large when the value is near to 0.8, medium if near to 0.5, and small if Ais near to 0.2. So, we can deduce 
that the used technique performs better than the baseline technique if its A value exceeds 0.5. 


4.3. Statistical test 

Even though the employed accuracy measures are helpful to compare different cost prediction 
models (i.e. checking if one method is better than the other), it remains necessary to perform the statistical 
test to assess the observed difference, i.e., it is significant or not [38]. We mention that we used the Wilcoxon 
test; it enables us to evaluate the difference between methods using their absolute error (AbE) (6). The 
significance level used is 0.05. 


5. RESULTS AND DISCUSSION 

We discuss in the present section all findings resulting from the assessment of the proposed method 
and the three compared techniques: the SVR-RBF-GS [17], the regression model of Zia’s work [20], the 
SVR-RBF-Weka which means the SVR-RBF evaluated using default values of the weka tool. First, we tune 
our model using the Optainet algorithm to get the best model. Second, we evaluate our method and the three 
aforementioned methods (used in comparison) using A and SA metrics. We reported the obtained values in 
Table 3. 


Table 3. The evaluation of each technique based on the SA and the effect size-the highest values are in bold 


Approaches SA A References 
SVR-RBF-Optainet 0.896 6.231 - 
SVR-RBF-GS 0.893 6.138 [17] 
Zia et al. regression 0.865 5.755 [20] 
SVR-RBF-Weka 0.195 1.261 - 


A large A value means that the assessed technique doesn’t produce prediction by chance. Also, a 
large value of SA means that the evaluated technique generates acceptable estimates. Therefore, in 
conformity with LOOCV and the values of Table 3, all techniques outperform the random guess (SA>0). 
Yet, The SVR-RBF-Optainet holds the highest SA value (SA=89.6%), and the SVR-RBF-Weka model holds 
the lowest one (SA=0.195) despite its better performance against random guessing. All techniques in Table 3 
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have a large Avalue (A>0.5) while SVR-RBF-Optainet retains the largest one (A=6.23). Overall, all 
generated estimates weren’t made by chance. 

We note the insufficiency of SA and A measures to draw any conclusions concerning the accuracy 
of our method in comparison to the others. Thus, we applied additional metrics to rank all used techniques 
throughout the Borda counting method as shown in Table 4. According to Table 4, we recognize the 
superiority of our method regarding others. The SVR-RBF-Optainet was ranked first, followed by 
SVR-RBF-GS while the SVR-RBF-Weka can be considered as the worst method for being classed last. In 
sum, the obtained results reveal that our proposed model, besides all compared methods, outperforms the 
random guess (SA>0). Also, SVR-RBF-Optainet is the best (ranked first). To make clear and justify the 
yielded conclusion, the statistical Wilcoxon test was performed. This test was made based on the absolute 
error AbE and with 0.05 as a significant level. Table 5 shows the resulting p-values. We notice that SVR- 
RBF-Optainet significantly outperforms the SVR-RBF-Weka. This result confirms its ranking obtained from 
the Borda count method. However, the difference in performance regarding the others (p-values) is not 
statistically significant. 


Table 4. The ranks of methods based on borda count 


Ranks Techniques References 
1 SVR-RBF-Optainet - 
2 SVR-RBF-GS [17] 
3 Zia et al. regression [20] 
4 SVR-RBF-Weka - 


Table 5. The p-values of methods based on statistical test. 
Methods — SVR-RBF-Weka__Zia et al. regression _SVR-RBF-GS 
p-values 0.0000 0.1228 0.7525 


To deeply analyze the performance of our proposed model, we compare it to many studies reported 
recently in literature which is the case for the following papers: [14], [17], [20]-[22], [39]. However, these 
articles applied various performance metrics and validation techniques. For instance, in [21], the authors 
didn’t mention the technique used for validation, and they calculated the Pred (8%) rather than the frequently 
employed prediction which is Pred (0.25). Also, in [39], the employed measures are MCMER and MMER 
rather than MdMRE and MMRE. Even though we attempted to compare our model to the previously cited 
studies by being consistent with the experimental design which is commonly used in cost prediction as shown 
in Table 6. 


Table 6. The comparison between methods based on the values of MAUMRE, MMRE, and Pred 


Techniques Pred (25%) MdMRE MMRE References 
SVR-RBF-Optainet (the proposed model) 100 0.0392 0.0596 - 
Pred (0.08)=71.428 0.0397: MdMER 0.0602: MMER 
SVR-RBF-GS 100 0.0426 0.0620 [17] 
Pred (0.08)=66.667 0.0408: MdMER 00,0613: MMER 
SVR-RBF-Weka 28.571 0.4710 0.4671 - 
SVR-RBF 95.9052 NA 0.0747 [14] 
Zia et al. regression model 57.14 0.0714 0.0719 [20] 
ABC-PSO 100 0.0333 0.0569 [21] 
Pred (8%)=66.667  0,0344: MdMER = _0,0564: MMER 
DT" 38.09 NA NA [39] 
SGB* 85.71 NA NA 
RF 66.67 NA NA 
GRNN™ 85.9182 NA 0.3581 [22] 
PNN™ 87.6561 NA 1.5776 
GMDH-PNN™ 89.6689 NA 0.1563 
CCNN™ 94.7649 NA 0.1486 


*DT: Decision Tree, SG: Stochastic Gradient Boosting, RF: Random Forest. 
**GRNN: General Regression Neural Network, PNN: Probabilistic Neural Network, GADH-PNN: GMDH PolynomialNeural 
Network, CCNN: Cascade-Correlation Neural Network. 


We observe from Table 6 that our model performs better than all compared models based on all 
metrics, apart from the ABC-PSO model in terms of MMRE, and MdMRE (where it has a slightly higher 
MdMRE (+0.006), and MMRE (+0.002)). Even if our model produces similar Pred (25%) than that of the 
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ABC-PSO, we observe that it outperforms it greatly based on the value of Pred (8%) with exactly 71.42% for 
SVR-RBF-Optainet against 66.67% for ABC-PSO) so with a remarkable improvement of (+4.76). We notice 
that the ABC-PSO method can be excluded from the comparison since the author didn’t mention which 
validation technique was employed. So, they might have used the whole dataset to train the model and test it 
as well. Finally, from our conducted analysis, we conclude that the Optainet-based SVR-RBF is a promising 
agile effort estimation model that can produce reliable and accurate estimates. 


6. CONCLUSION 

The present work proposed an enhanced model for ASDCE purposes. This model is based on the 
optimized ainet that tunes the parameters of the SVR-RBF model. We assessed our model's performance and 
made a detailed comparison analysis utilizing the trusted protocol that is based on: (SA, A, Borda counting 
and significance test) and by using the most used criteria, which are the Pred (25%), MdMRE, and MMRE. 
The findings indicate the superiority of our method regarding others. According to the statistical test results, 
the SVR-RBF-Optainet was ranked first based on the Borda counting method, and it significantly 
outperforms the SVR-RBF. Additionally, The SVR-RBF-Optainet outperforms the SVR-RBF regarding the 
Pred (25%) (+0.42) and MMRE (-0.1). Moreover, our model performs better than all compared models based 
on all metrics, apart from the ABC-PSO model in terms of MMRE, and MdMRE. In addition, the 
SVR-RBF-Optainet outperforms the SVR-RBF-GS, and all SVR models using different kernel optimization. 
Besides, the SVR-RBF-Optainet outperforms greatly Zia’s regression based on the MdMRE, MMRE, and 
Pred. Also, it performs better than the ABC-PSO regression technique in terms of Pred (8%). Finally, we can 
conclude that the Optainet based SVR-RBF model is a powerful technique. However, we are unable to 
confirm its superiority in all situations. So, further research should be done using various datasets to 
generalize the results. 
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